The hypertext transfer protocol (HTTP) streaming of media, in particular video, has lately evolved beyond simple progressive download to allow adaptability and live content. In progressive download, which is currently the dominant manner in which video is consumed on the Internet, a client computer or software application downloads (fetches) a big file being assisted by a server and plays out the downloaded content in parallel with downloading. The HTTP streaming is based on splitting the media content (i.e., the file) into multiple segments, each corresponding to a small time interval of content (e.g., 10 seconds of playing out content). The client is provided with a manifest file containing a list of the segments and where from to download them (e.g., URL addresses), thus, enabling the client to download the segments one by one.
An example of HTTP streaming is Adaptive HTTP streaming that has been defined by the Third Generation Partnership Project (3GPP) in, for example, current versions of 3GPP Technical Specification (TS) 26.234 and 3GPP TS 26.244. A counterpart of 3GPP Adaptive HTTP streaming has been defined in the current version of Open Internet Protocol Television Forum (OIPF) in its version-2.0 specifications.
Underlying architecture of systems on which the above-specified protocols may be executed is defined, for example, as Internet Protocol (IP) Multimedia Subsystem (IMS) in 3GPP TS 23.228 V8.4.0, IP Multimedia Subsystem (IMS) Stage 2 (Release 8), March 2008, and other current versions of TS 23.228. IMS is described in, for example, R. Noldus et al., “Multi-Access for the IMS Network”, Ericsson Review No. 2, pp. 81-86 (2008); U. Olsson et al., “Communication Services—The Key to IMS Service Growth”, Ericsson Review No. 1, pp. 8-13 (2008); and P. Arberg et al., “Network Infrastructure for IPTV”, Ericsson Review No. 3, pp. 79-83 (2007). Approaches to IMS-based IPTV are described in M. Cedervall et al., “Open IPTV Forum—Toward an Open IPTV Standard”, Ericsson Review No. 3, pp. 74-78 (2007), and T. Cagenius et al., “Evolving the TV experience: Anytime, Anywhere, Any Device”, Ericsson Review No. 3, pp. 107-111 (2006).
A user equipment (UE), such as a mobile phone, a set-top box (STB), a TV having STB capabilities, etc., can access services offered through an IMS in many ways, both wired (e.g., Ethernet, cable modem, digital subscriber line, etc.) and wireless (e.g., 3GPP-specified cellular radio, IEEE 802.11, IEEE 802.16, etc.). In general, IPTV is a term used for a system receiving and displaying multimedia streams encoded as series of IP data packets. An IPTV system and media bookmarks for such a system are described in International Publication WO 2010/016836 by N. Mitra et al.
In contrast to streaming servers based on protocols like the real-time streaming protocol (RTSP) and real-time transport protocol (RTP), in the IPTV methods of streaming content indicated above, a big content file is split into different segments/files that are downloaded via a standard web protocol (such as HTTP). This type of protocol is considered “cache-friendly”, or Content Distribution Network (CDN) friendly, because no particular state in the server or cache is required. These methods enhance flexibility by allowing changes in the time intervals and segments. For example, an advertisement segment can be inserted by changing the content during one interval (which is called a “period” in the 3GPP specifications), and then returning to the regular content according to the manifest file in the next interval.
Adaptivity is also achieved by providing multiple versions (which are called “representations” in the 3GPP specifications) of the content for a time interval, so that the client can choose to download the version most suitable for the client's resources (given the network performance/download time) and preferences.
The media players discussed here may be either full-screen players, or may be embedded in an application, or in a browser, for example using the new HTML-5 video element. The application program interfaces (APIs) can be either native or Javascript APIs.
In the conventional systems, clients receive information from an interface comprising a set of functions and variables related to the downloading and the playing of the download media. In OIPF version 2, vol. 5, section 7.14.9, the following properties and methods are supported on audio and video objects as defined in Section 5.7.1 of the current version of CEA-2014-A.
function onReadyToPlay( )The function that gets called when enough (as determined by the OITF) of the media after the current play position has been buffered to start/continue playback.The specified function shall be called with no arguments.This event SHALL be generated whenever there is a state transition between state 4 (“buffering”) and state 1 (“playing”). The event SHALL also be generated at the moment that enough data has been buffered to start playback, whilst in state 2 (“paused”).
Here OITF is defined in the functional architecture specification as a block which resides inside a network offering IPTV services. OITF provides the functionality necessary to access the IPTV services in the network. In these definitions reproduced from the OIPF version 2, vol. 5, section 7.14.9, terms that are written with capital letters (e.g., “SHALL”) are to be interpreted according to RFC 2119 “Key words for use in RFCs to Indicate Requirement Levels”, published in March 1997 by the Internet Engineering Task Force (IETF) (which describes methods, behaviors, research, or innovations applicable to the working of the Internet and Internet-connected systems). Description of the properties and methods in OIPF version 2, vol. 5, section 7.14.9 continues below.
Boolean readyToPlayProperty that can be used to inspect whether or not enough (as determinedby the OITF) of the media after the current play position has been buffered to start playback.Returns true if enough data has been buffered. Returns false if not enough data has been buffered.
Integer getAvailablePlayTime(Boolean fromPlayPosition)DescriptionReturns how much content is available for playback.If argument fromPlayPosition has value true, this method returns an estimate of how much data in milliseconds is available in the buffer for play back after the current play position.If argument fromPlayPosition has value false, this method returns an estimate of the total buffer length in milliseconds (i.e. this includes all data available in the bufferbefore and after the current play position).Arguments fromPlayPosition Indicates whether the available play time should be calculated from the current play position onwards, or from the start of the buffer.
Boolean setBufferingStrategy(String name)DescriptionRequest to change the buffering strategy. Valid values for argument name include:“sustained_playback”: this is the default strategy, whereby the incoming videostream should be rendered with as little hiccups or lost frames as possible. This meansthat the buffering threshold for triggering an onReadyToPlay event is chosen to besufficiently large to deal with variations in network throughput.“low_latency”: this is a strategy whereby the incoming video stream should berendered with an as low as possible latency between receiving the content and theactual playback of the content. This means that buffering threshold for triggering anonReadyToPlay event needs to be made sufficiently small in order to playback thecontent as soon as possible after it has been received.The default strategy if the method is not called is “sustained_playback”.This method can be called during any play state, including play state 1 (‘playing’).This method returns true if the buffering strategy has been successfully changed tothe preferred buffering strategy. The method returns false if the buffering strategyhas not been successfully changed.If the OITF does not distinguish between the two modes, the method returns false.
Additionally, when the function onReadyToPlay is executed, a corresponding Document Object Model level 2 event SHALL (as defined in Worldwise Web Consortium—W3C—current documents) is generated in the following manner:
Corresponding DOM 2 EventIntrinsic eventDOM 2 eventpropertiesonReadyToPlayReadyToPlayBubbles: NoCancelable: NoContext Info: None
These DOM 2 events (as defined in W3C “Document Object Model (DOM) Level 2 events”, version 1 from 2000) are directly dispatched to the event target, and will not bubble nor capture. Applications should not rely on receiving these events during the bubbling or the capturing phase. Applications that use DOM 2 event handlers SHALL call the addEventListener( )method (which described in current version of W3C, “Scalable Vector Graphics Tiny 1.2 Specification”) on the CEA-2014 A/V Control object. The third parameter of addEventListener (i.e. “useCapture”) is to be ignored.
Some media players using progressive download (e.g., a typical YouTube player) display a time line with a current play time point, and a progress bar with a buffering point showing how much content has been downloaded (i.e., buffered). The time line and the progress bar showing equivalent time may be overlapped. The media players typically begin to play the content after a predetermined amount of content has been downloaded and buffered. Then, the players continue downloading parallel with playing out the content. If the current play time point reaches the buffering point (i.e., all the downloaded content has been played), the playing is interrupted until enough content (e.g., a few seconds worth of played content) is downloaded and buffered. This situation, which is visually recognizable when the time line and the progress bar are displayed, is a well-known problem for the conventional media players.
Existing adaptive streaming solutions are designed to provide an interruption-free play-out experience, but the quality varies if the network capacity varies. These adaptive streaming solutions are often also used to replace progressive download for live content, such as, a sports event. Moreover, third-party applications for mobile devices use adaptive streaming solutions for all video content longer than a predetermined duration (e.g., 10 minutes) independently of whether the content is live or not.
Conventional multimedia players are not capable to provide streaming free of interruptions and at a constant quality. FIG. 1 is a graph illustrating the evolution of a played out bandwidth (i.e., bandwidth as a function of time) for a conventional system. Data is buffered only from a time t=0 until a moment marked as t1 in FIG. 1. Multiple changes in the play-out bandwidth data and, thus, in the viewing quality occur at t2, t3, t4, etc.
However, especially for video-on-demand, the end-user (viewer) or the application developer may still want to have the progressive-download user experience, where the quality is not varying, but the data is instead buffered longer.
Additionally, the end-user/developer may also want (and the conventional multimedia players do not provide for) to be able to set an upper limit on the bit-rate (bandwidth), especially when the end-user pays traffic charges depending on the traffic amount.
Another desirable feature, which is not provided by the conventional media players, is providing an indication of the current media quality (bandwidth or bit-rate). For example, presence of a bit-rate (or bandwidth) meter overlaying the content and showing the current bit-rate (or bandwidth) and its evolution would enable the end-user to understand when the variations in quality are due to network variations, and when are due to inherent problems in the media player, or on the server. Information and control related to downloading and/or playing out would lead to a more consistent user experience, which is very important especially for paid services.
Accordingly, it would be desirable to provide methods and devices that overcome the afore-described problems and drawbacks of the conventional media players.